b — t unification and large atmospheric mixing: a case for non-canonical see-saw 
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We study the second and third generation masses in the context of the minimal renormalizable 
SO(10) theory. We show that if the see-saw takes the non-canonical (type II) form, large atmospheric 
neutrino mixing angle requires b — t unification. 
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A. Introduction. A suspected quark- lepton sym- 
metry is, as we know, badly broken by the difference in 
their mixing angles. Small Vckm mixing should be con- 
trasted with the maximal mixing for atmospheric neutri- 
nos and probably large mixing for solar neutrinos. Why 
is this so? This has become one of the major issues in 
the so-called fcrmion mass and mixing problem. 

In this Letter we address this question in the min- 
imal renormalizable SO (10) theory, without any addi- 
tional symmetries or interactions. We focus only on the 
second and third generations for three reasons: 

(i) in this case the neutrino mixing angle is maximal 
and experimentally established; 

(ii) it is much likely that in the case of the first family 
we can not ignore higher dimensional operators; 

(iii) in this simple 2x2 case we can actually present 
analytic expressions. 

Our main result is the following. We show that in the 
case of non-canonical see-saw, large neutrino mixing an- 
gle requires b — r unification. The rest of the paper is a 
proof of this statement and a discussion of its implica- 
tions. 

The choice of SO(10) theory is highly natural. It uni- 
fies a family of fermions; it unifies their interactions (ex- 
cept for gravity) ; it has a see-saw mechanism Q| of small 
neutrino mass naturally built in; it has charge conjuga- 
tion as a gauge symmetry; and, in its supersymmetric 
version, leads naturally to a theory of R-parity (^||. 

The last result holds true in the renormalizable version 
of the theory with a 126 h dimensional Higgs supermul- 
tiplet used to give masses to the right-handed neutrinos. 

B. Canonical (type I) versus non-canonical (type II) 
see-saw mechanism. The minimal Higgs that breaks 
SU(2)xU(l) symmetry and gives mass to the fermions is 
under the Pati-Salam SU(2) L xSU(2) fl xSU(4) c symme- 
try 
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H 



(2, 2,1) + (1,1, 6), 



(1) 



and so (10//) = ((2,2,1)) ^ implies the well-known 
quark-lepton symmetric relation for fermion masses 



m D = m E , 



(2) 



which works well for the 3 rd family, and fails badly for the 
first two. You can correct this by adding more Higgses, 
or appealing to higher dimensional operators (see for ex- 
ample |Q,P,bl). However, a nice and important point was 
raised around twenty years ago |7). Ten years ago Babu 
and Mohapatra utilized it to study neutrino masses and 
mixings M. With 10// and 126 h the Yukawa sector of 
the Lagrangian is given by 



(3) 



where tp stands for the 16 dimensional spinors which in- 
corporate a family of fermions, and Y\q and Y126 are the 
Yukawa coupling matrices in generation space. 
From 

126// = (3, 1,10) + (1,3, TO) + (2, 2, 15) + (1,1, 6) (4) 
one has 



*i 26 ((l,3,10)i 



(5) 



where ((1, 3,TU)i26) = Mr, the scale of SU(2)^ gauge 
symmetry breaking. 

It can be shown that, after the SU(2)xU(l) breaking 
through (10 H ) = ((2,2,1)) rj M w , the (3,1,10) multi- 
plet from 126// gets a small vev P,|l0| 



((3,1,10)126) « 



M, 



parity 



(6) 



where M par i ty is the scale of the breakdown of parity. 
In general Mr and M par it y are not necessarily equal, 
but typically one breaks parity through the breaking of 



SU(2)/j symmetry, in which case Mr = M par i ty . 
what we take hereafter. 

In turn, neutrinos pick up small masses 



M„ 



Y 126 ((3, 1, 10) i 26 ) + m T D M-^m D 



This is 



(7) 



where mu is the neutrino Dirac mass matrix. It is often 
assumed, for no reason whatsoever, that the second term 
dominates. This we call canonical (often called type I) 
see-saw. In what follows we explore the opposite case, 
which we call non-canonical (type II) see-saw. After all, 
it does not involve Dirac mass terms and so there is no 
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reason a priori in this case to expect quark- lepton analogy 
of mixing angles. In this sense the non-canonical see-saw 
is physically more appealing. More than that, we will 
show that the large leptonic mixing fits perfectly with 
the small quark mixing, as long as nib — m T . 

The crucial ingredient is the fact J^] that through a 
non- vanishing tadpole a (2, 2, 15) field in 126/r also picks 
up a vev: 



((2,2,15)i2a) 



/ M 



GUT 



((2,2,l)io) 



(8) 



In the supersymmetric version of the theory this re- 
quires a 210 dimensional Higgs at the GUT scale. 

C. Non-canonical see-saw: b — r unification and large 
atmospheric neutrino mixing. Most of the study 

throughout the years assumed the canonical see-saw, i.e. 
the second term dominates in ([?]). The original claim of 
H that the leptonic mixing matrix Vi had a small 2 — 3 
element was questioned by using a non-minimal model 
p| or the freedom to adjust the phases in the mixing 
matrices Q. Last year we studied the opposite 
case, the non-canonical see-saw and noticed that it fitted 
nicely with a large 2 — 3 mixing angle responsible for 
atmospheric neutrinos. 

We give here a simple argument in favour of this. We 
show how maximal fx — r mixing fits nicely with b — t 
unification. 

To see this, notice that fermion masses take the follow- 
ing form 



M l7 = Y W V? + Yl26f 126 . 

M D = Y w v( + Y 126 vf 26 , 



M E = Y w vf - 3yi 2 6uf 26 
Mtf = Yiaa((3, MOjiae) , 



(9) 
(10) 

(11) 
(12) 



where U, D, E, N stand for up quark, down quark, 



charged lepton and neutrino, respectively, while and 
are the two vevs of (2, 2, 1) in 10# and (2, 2, 15) 



y 126 



in 126/f, and the last formula is the assumption of the 
non-canonical see-saw. The result is surprisingly simple. 
Notice that [HI 



M N oc Yi2 6 <xM D -M E 



(13) 



Now, let us study the 2 nd and 3 rd generations, and 
work in the basis of Me diagonal. The puzzle then is: 
why a small mixing in Md corresponds to a large mixing 
in Mjv? For simplicity take the mixing in Md to vanish, 
6d = 0, and ignore the second generation masses, i.e. 
take to, = m„ = 0. Then 



M N oc 





my, — m T 



(14) 



Obviously, unless nib = Tn T , neutrino mixing vanishes. 
Thus, large mixing in Mn (the physical leptonic mixing 



in the above basis) is deeply connected with the b — r 
unification. Notice that we have done no model building 
whatsoever; we only assumed a renormalizable SO(10) 
theory and the non-canonical see-saw. 

Before we discuss (^) more carefully by switching on 
irifj,, m s and the mixings, let us comment on the implica- 
tion of our result. First, notice that it does not depend 
on the number of 10// 's. Notice also that it is not easily 
generalized to three generations, i.e. it is not easy to give 
the same reasoning why the solar neutrino mixing should 
be large (to be confirmed experimentally). 

In short, our results should be taken as an argument 
in favour of the non-canonical see-saw: large atmospheric 
mixing angles and b — t unification seem to prefer clearly 
this form of the see-saw mechanism. 

D. Quantitative analysis. Let us now be more 

quantitative and turn on m s , to m and 9d- Notice that Qo 
is not a 2 — 3 Vqkm mixing angle, but rather a difference 
between charged lepton and down quark mixing angles 
(recall that we choose Me diagonal). 

An important comment. It is not that all the 32 dou- 
blets in (2, 2, 1) and (2, 2, 15) remain light; with the min- 
imal fine-tuning we end up with only two of them at Mz- 
Let us denote their vevs by Vi (i — u,d), where My/ = 
g\/v 2 + and we adopt as usual tan j3 = v u /vd- Then 
we can write 

v{ = Vi cos en , v\ 2e = Vi sin on , (i = u,d) , (15) 
where aj are unknown angles. Defining 

(16) 



tan a u cos ad 
x = , y = 



tan ad 



cos a,. 



(notice that either x 2 < y 2 < 1 or x 2 > y 2 > 1), it is a 
simple exercise to derive from (p|)-(|l2|) 



Y E = z [iyYu - (3 + x)Y D ] , 

1 — x 

Y N =c{Y E - Y d ) , 



(17) 
(18) 



where M v = v u Y v , M D = v d Y D , M E = v d Y E , M N oc 
Yjsr, and c is an unknown constant in this theory. Since 
Y's are symmetric, we can write for species X 



Y x = XY«X 



(19) 



where Y£ are diagonal Yukawa matrices and X are in 
general unitary. In what follows we do not wish to play 
with the adjustment of phases and so take X to be or- 
thogonal matrices for simplicity and transparency. 

Let 9i, 9d and q denote the rotation angles in E T N, 
D T E and D T U respectively (0/ and 9 q are the leptonic 
and quark weak mixing angles respectively). From ( p^ ) 
we get 



tan 20i = 



sin 26 D 



Vb-Vs 



cos 29 d 



(20) 
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Next, we 
the dependence 
cos 8d, c q — cos9 q , etc. 



wish to connect dp with 6 a in order to have 
3nce of 9i with 9 q . ^From ( |17| ) one has (cd = 



( c 2 D y T 

\s 2 D y T 



Vb 

Vs 



C 2 gVt 



■s 2 q yc 



( x 



c 2 dVt 
s 2 dVt 



- s 2 dVv 

- c2 dVv 



- tyb 
-3y s 



After introducing 



Vc Vs Vn Vb - Vr 
— , £d = — ; e e = — , e— 

Vt Vb Vr Vb 



(21) 



(22) 



and after some computational tedium we get from (|]j 
and (|2l 



(1 - e e ) tan 9 D [(1 - e u e d ) tan 2 9 q 



-ib - ran- c 9 -f (e„ - e d )] = 
(1 - e„) tan 0, [(1 - e e e d ) tan 2 9 D + (e e - e d )] . (23) 



In the limit ei — (i — u, d, e) there are two solutions: 
tan 9d — and tan 6*0 = tan 9 q . The first solution can be 
shown to be unrealistic, whereas the second one gives the 
important relation between the physical mixing angles of 
quarks and leptons: 



tan 26, 



sin 20 „ 



2 sin 2 9„ 



(24) 



Since 9 q — bc of VckMi 8 q ~ 1CT 2 , (|24|) shows man- 
ifestly that tan#/ rj 1 requires e ~ 0, i.e. ?/{, ~ y r as we 
argued repeatedly. 

Let us now switch on the second generation masses, 
i.e. let us take £j ^ 0. From ( p3| ) one can see that the 
physically acceptable solution is 



, tan 9„ 



(25) 



From (pG) is then obvious that b — r unification y T = 
Vb + 0(6)is sufficient to make the mixing angle large, 
i.e. tan 20; = 0(1) ^ 6. This is our main result, rather 
nontrivial in our opinion. A small quark mixing angle 
automatically leads to a large leptonic mixing in the 2 — 3 
case. 

E. From high to low energy: running. 

Our expressions are valid at the unification scale 
Mgut- Thus we must run the physical parameters from 
Mgut to Mz in order to be precise. However, in this 
case the running is not so important as it may seem. 
Namely, in this letter we want to study the implications 
of the SO(10) symmetry (in its minimal renormalizable 
version) on fermion masses and mixings. What we said 
up to now is equally valid in ordinary and supersymmet- 
ric (with 210# Higgs) SO(10) gauge theory. We wish to 
emphasize the generic feature of the model, that is the 
connection between the large 9 a t m and b — r unification 
and do not worry so much about the precise numerical 
estimates. This requires specifying precisely the nature 



of the low energy effective theory. Still, it is instructive 
to see the impact of running. We thus discuss briefly the 
supersymmetric case and leave the complete discussion 
for a longer paper now in preparation. 

The neutrino matrix elements My run at the 1- 
loop level and neglecting threshold effects according to 

@0B 



6s! - \gl 



Mi. 



(26) 



where t = In (Q/Mz), gi is normalized in the SU(5) fash- 
ion, i, j — 2,3 stand for the second and third generations, 
and hi =0, ^3 = 1. The neutrino mixing angle at the 
electroweak scale is 



tan 29, 



l\Mz 



2M 23 (0) 



M 22 (0)-M 33 (0) 

2M 23 (t GUT )B T 



M 22 (t G uT) - B*Mas(t auT ) ' 
where tQjjT = In (Mgut/Mz) and 
1 



exp 



16tt 2 



Vr(t)dt 



(27) 



(28) 



The elements Mij(tauT) are exactly the ones discussed 
throughout the paper. We can thus recalculate j2fj| ) 
(valid at Mgut) at Mz'- 



tan 26 1 



B T sin26' c , 



Mz 



±±^-cos26>d 



(29) 



All the parameters of the right-hand-side are to be 
evaluated at the GUT scale. For this reason the same 
equation ( p3|) is again used to express 6d- Clearly, as 
before, large neutrino mixing angle comes out as soon as 
Vb and y T unify at the GUT scale. Of course, the precise 
value of the neutrino mixing angle depends on this run- 
ning, however the qualitative behaviour does not change. 

A more detailed approach would require to use numer- 
ical techniques to account for (1) the running as func- 
tion of tan/3; (2) the inclusion of threshold corrections 
]l8[ ; and (3) first generation effects. However, threshold 
effects in SO(10) are bound to be important and high 
precision calculations may actually not be so useful, see 
for example fl9|| . 

What about the values of neutrino masses? We do not 
enter into this issue here since we have no new results 
beyond Q. 

F. Summary and outlook. 

The sharp contrast of quark and lepton mixings is often 
considered a deep puzzle. We argued here that it is actu- 
ally quite natural in the minimal SO(10) renormalizable 
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the ory. All that is required is that the see-saw mechanism janovic, Nu cl. Phys. B 597 (2001) 89 [arXiv: jiep 



take s a non-canonical form free from Dirac masses. The 
approximate formula ([24|) expresses it clearly: a small 
quark mixing 9 q = d(, c m 1(U 2 gives naturally a large 
@l = 6atm if e ~ 0, i.e. w y T . Actually, the essence 
of our work lies in formulae (|l3|)-([l4|). Formula (14), 
valid in the approximation of vanishing second generation 
masses and vanishing quark and lepton mixings speaks 
eloquently: unless mf, = m T at the large scale, we will 
have a vanishing atmospheric neutrino mixing. In short, 
the non-canonical see-saw marries nicely b — r unification 
with the maximal atmospheric neutrino mixing. This can 
be of great help in trying to pin-point the nature of the 
see-saw mechanism: our study points in favor of the non- 
canonical version. 

Strictly speaking, a numerical study showed that in the 
3x3 case, by playing with CP phases, even the canonical 
see-saw can be made to work [jl2). However, in our case, 
the 2 — 3 family study offers physical insight into the 
question, and after all the first family of fermions may 
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suffer from the higher dimensional operators. The 10# 
and 126 h, the minimal Higgses needed to give masses [14 
to all fermions, work beautifully: 10# offers mj, = m T , 
and 126/f offers 3m s = — m M at the GUT scale; and in [15 
this framework a small 9 c b (0 ts ) and a large 9 a tm become 
naturally connected. Thus, the observational evidence [16 
that quarks and leptons have sharply different mixing 
angles fits nicely with the belief that they are one and [17. 

the s arn ^ oblect at a fnmrlamPTital lmr^l 



We are grateful to Rabi Mohapatra for his encourage- 
ment, and to Alejandra Melfo for useful comments and 
a careful reading of the manuscript. The work of B.B. 
is supported by the Ministry of Education, Science and 
Sport of the Republic of Slovenia. The work of G.S. is 
partially supported by EEC, under the TMR contracts 
ERBFMRX-CT960090 and HPRN-CT-2000-00152. We 
express our gratitude to INFN, which permitted the de- 
velopment of the present study by supporting an ex- 
change program with the International Centre for Theo- 
retical Physics. 



[18 
[19 



C. H. Albright, K. S Babu and S. M. Barr , Phys. Rev. 
Lett. 81 (1998) 1167 |arXiv:hep-ph/9802314 1. 
K. S. Bab u, J. C. Pati and F. Wi lczek, Nucl. Phys. B 566 
(2000) 33 |arXiv:hep-ph/981253£ l. 

T. Blazek, R. Dermisek and S. Raby , Phys. Rev. D 65 
(2002) 115004 ]arXiv:hep-ph/0201081 |. 
G. Lazarides, Q. Shafi and C. Wetterich, Nucl. Phys. B 
181 (1981) 287. 

K. S. Babu and R. N. Mohapatra, Phys. Rev. Lett. 70 



(1993) 2845 [arXiv:hep-ph/92092L 



R. N. Mohapatra and G. Senjanovic, Phys. Rev. D 23 
(1981) 165. 

M. Magg and C. Wetterich, Phys. Lett. B 94 (1980) 61. 
K. y. Oda, E. Takasugi, M. Ta naka and M. Yoshimu ra, 
Phys. Rev. D 59 (1999) 055001 fcrXiv:hep-ph/9808241 1 . 
K. Matsuda, Y. Koide, T. Fukuyama and H. Nishiura, 
Phys. Re v. D 65 (2002) 033008 [Erratum-ibid. D 65 (2002) 
079904] ]arXiv:hep-ph/0108202t . 

B. Bajc, G . Senjanovic and F. Vissani, [arXiv: jiep 
ph/0110310[ 



B. Brahmacha ri and R. N. Mohapatr a, Phys. Rev. D 58 
(1998) 015001 |arXiv:hep-ph/9710371 |. 
P. H. Chan kowski and Z. Plucien nik, Phys. Lett. B 316 
(1993) 312 |arXiv:hep-ph/9306333 1 . 

K. S. Babu, C. N. Leung and J. Pantale one, Phys. Lett. B 

319 (1993) 191 |arXiv:hep-ph/9309223 |. 

K. R. Balaji, A. S. Dighe, R. N. Mohapatra and M. K 

Parida, Ph ys. Rev. Lett. 84 (2000) 5034 |arXiv:hep- 

ph/0001310| ]. 

See, e.g. L. J. Hall , R. Rattazzi and U. S arid, Phys. Rev 



D 50 (1994) 7048 fcrXiv:hcp-ph/930630£ ] . 
V. V. Dixit and M. Sher, Phys. Rev. D 40 (1989) 3765. 



[1] M. Gell-Mann, P. Ramond and R. Slansky, proceedings of 
the Supergravity Stony Brook Workshop, New York, 1979, 
eds. P. Van Niewenhuizen and D. Freeman (North-Holland, 
Amsterdam); T. Yanagida, proceedings of the Workshop 
on Unified Theories and Baryon Number in the Universe, 
Tsukuba, Japan 1979 (edited by A. Sawada and A. Sug- 
amoto, KEK Report No. 79-18, Tsukuba); R. N. Mohapa- 
tra and G. Senjanovic, Phys. Rev. D 23 (1981) 165. 

[2] C. S. Aulakh, A. Melfo, A. Rasin and G. Senjan ovic, Phys. 
Lett. B 459 (1999) 557 |trXiv:hep-ph/9902409 ] . 

[3] C. S. Aulakh, B. Bajc, A. Melfo, A. Rasin and G. Sen- 



4 



